Repository navigation
feat(admission): add opt-in shared concurrency admission leases - #29
Conversation
Add a Durable Object admission authority with per-principal and per-model-group concurrency limits, renewable leases and Flask/Worker clients behind ADMISSION_ENABLED, default off. The class is inert while disabled.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ❌ Deployment failed View logs |
multillm-proxy | 4497f7a | Oct 08 2026, 09:51 PM |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4497f7a056
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const [count] = this.rows(`SELECT COUNT(*) AS n, MIN(expires_at) AS next_expiry FROM admission_leases ${where}`, ...args); | ||
| if (count.n >= maximum) return failure("admission_denied", 429, Math.max(1, Math.ceil((count.next_expiry - now) / 1000))); |
There was a problem hiding this comment.
Compute Retry-After for the lease that frees capacity
When a configured limit is lowered below the number of existing leases, MIN(expires_at) only identifies the first lease to expire, which may still leave the scope at or above its new limit. For example, reducing a principal from four active leases to a limit of one returns the first expiry even though three leases remain afterward, so clients following Retry-After are denied again. Select the expiry that reduces the count below maximum rather than always using the minimum.
Useful? React with 👍 / 👎.
| const [count] = this.rows(`SELECT COUNT(*) AS n, MIN(expires_at) AS next_expiry FROM admission_leases ${where}`, ...args); | ||
| if (count.n >= maximum) return failure("admission_denied", 429, Math.max(1, Math.ceil((count.next_expiry - now) / 1000))); |
There was a problem hiding this comment.
Compute Retry-After for the lease that actually frees capacity
When a configured limit is lowered below the number of existing leases, MIN(expires_at) only identifies the first lease to expire, which may still leave the scope at or above its new limit. For example, reducing a principal from four active leases to a limit of one returns the first expiry even though three leases remain afterward, so clients following Retry-After are denied again. Select the expiry that reduces the count below maximum rather than always using the minimum.
Useful? React with 👍 / 👎.
| except AdmissionError as error: | ||
| self.failure = error | ||
| # Transport cancellation can wait for readers that call check(); never hold their lock. | ||
| self.release() |
There was a problem hiding this comment.
Cancel upstream before waiting on release
When renewal fails because the authority connection is stalled, this synchronous release immediately makes another call to the same authority before invoking on_lost; that second call can consume the full connect/read timeout while the upstream generation continues running without receiving its cancellation signal. Invoke the cancellation collaborator before, or concurrently with, the best-effort release so an authority outage does not add another timeout of provider work.
Useful? React with 👍 / 👎.
10c1c9b
into
integrate/gateway-waves-1-2
Summary
Adds shared concurrency admission behind
ADMISSION_ENABLED, which defaults to false. While the flag is off, no lease call or Durable Object access happens and dispatch, responses, headers and storage are unchanged. An emptyADMISSION_LIMITS_JSONmeans unlimited even when the flag is on; there is no default gate on any model.AdmissionCoordinatorDurable Object, using SQLite storage, keeps expiring leases across 64 shards. Native Worker and Flask callers therefore compete for the same capacity.ADMISSION_LIMITS_JSONsets per-principal and per-model-group concurrency limits, for example{"principal": 4, "model_groups": {"heavy": 1}}.Retry-After, measured until the earliest blocking lease expires.POST http://intelligence.internal/v1/admissionis reachable only through Container egress, never from the public router.ADMISSION_AUTHORITY_URL; a process-local semaphore is never presented as global.Details are in
docs/admission-leases.md.Deployment
wrangler.jsoncadds theADMISSION_COORDINATORbinding and an additivev3_admissionDurable Object migration.wrangler deployapplies it on the next push to main. Existing bindings and migration tags are unchanged, and the class makes no storage writes and schedules no alarms while disabled. No D1 migration is needed.Integration
This branch exports the class from
cloudflare-worker.mjsand teaches the test loader the new import. The Wave 3 integration PR adds the remaining wiring:Shared registrations
ADMISSION_ENABLED,ADMISSION_LIMITS_JSONandADMISSION_AUTHORITY_URLto the Container env keys.tests/test_admission_leases_worker.mjstotest:worker.worker/admission-do.mjsto the secret firewall inventory.Tests
tests/test_admission_leases.pyandtests/test_admission_leases_worker.mjs: new.npm run test:worker: 1,239 passed.cf:migrations:check,test:deploy, free routes, client headers and thewrangler deploy --dry-runbundle all pass.tests/test_frontend_build.mjsfails locally because@tailwindcss/cliis not installed. It fails the same way on main.This is part of Wave 3. It is based on the Wave 2 integration branch (#25), and the Wave 3 integration PR adds the README line.